Gaze tracking

ABSTRACT

According to an example aspect of the present invention, there is provided an apparatus comprising at least one processing core, at least one memory including computer program code, the at least one memory and the computer program code being configured to, with the at least one processing core, cause the apparatus at least to determine a reference point in a three-dimensional space based at least in part on locations of first and second features of a user&#39;s eye and on the user&#39;s gaze distance, and perform a mapping of the reference point into a viewed scene of a near-to-eye optical device to obtain an estimated gaze point and/or gaze direction of the user who is using the near-to-eye optical device, the mapping being based at least in part on calibration information associated with the user.

FIELD

The present disclosure relates to determining a direction and/or targetof a user's gaze.

BACKGROUND

Determining what a user looks at is of interest in a number of differentfields. For example, a disabled person may be equipped with a gazetracker enabling her to input characters to form words and sentences. Asanother example, an ambulance driver may be enabled to safely operateequipment of his ambulance or a nuclear power station operator may havehis gaze tracked to detect episodes of fatigue during a shift.

Gaze tracking may be performed based on a number of differenttechniques, which have as a common feature that the position of one eye,or both eyes, is measured to obtain input information to the gazetracking system to control a device, for example.

SUMMARY

According to some aspects, there is provided the subject-matter of theindependent claims. Some embodiments are defined in the dependentclaims.

According to a first aspect of the present disclosure, there is providedan apparatus comprising at least one processing core, at least onememory including computer program code, the at least one memory and thecomputer program code being configured to, with the at least oneprocessing core, cause the apparatus at least to determine a referencepoint in a three-dimensional space based at least in part on locationsof first and second features of a user's eye and on the user's gazedistance, and

perform a mapping of the reference point into a viewed scene of anear-to-eye optical device to obtain an estimated gaze point and/or gazedirection of the user who is using the near-to-eye optical device, themapping being based at least in part on calibration informationassociated with the user.

According to a second aspect of the present disclosure, there isprovided a method comprising determining a reference point in athree-dimensional space based at least in part on locations of first andsecond features of a user's eye and on the user's gaze distance, andperforming a mapping of the reference point into a viewed scene of anear-to-eye optical device to obtain an estimated gaze point and/or gazedirection of the user who is using the near-to-eye optical device, themapping being based at least in part on calibration informationassociated with the user.

According to a third aspect of the present disclosure, there is provideda non-transitory computer readable medium having stored thereon a set ofcomputer readable instructions that, when executed by at least oneprocessor, cause an apparatus to at least determine a reference point ina three-dimensional space based at least in part on locations of firstand second features of a user's eye and on the user's gaze distance, andperform a mapping of the reference point into a viewed scene of anear-to-eye optical device to obtain an estimated gaze point and/or gazedirection of the user using the near-to-eye optical device, the mappingbeing based at least in part on calibration information associated withthe user.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates an example system in accordance with at least someembodiments of the present invention;

FIG. 1B illustrates coordinate mapping in accordance with at least someembodiments of the present invention;

FIG. 2 illustrates an example apparatus capable of supporting at leastsome embodiments of the present invention;

FIG. 3 is a flow graph of a method in accordance with at least someembodiments of the present invention, and

FIG. 4 illustrates an example method of determining a gaze distance.

EMBODIMENTS

Disclosed herein are gaze tracking methods for near-eye optical devices,such as ocular devices, such as microscopes and binoculars. To obtaindependable gaze tracking in an ocular device which is invariant, atleast partly, to movement of the user's head, a reference point isdetermined in a three-dimensional space based on the direction anddistance of the gaze of the user, and the reference point is mapped to apoint in viewed scene based on a calibration matrix obtained from theuser in a calibration process. Therefore, advantageously,characteristics of a coordinate transformation between a coordinatesystem of an eye-tracking sensor, such as a camera, and a coordinatesystem of the user's viewed scene, which may be obtained with a scenecamera, for example, need not be determined. The scene viewed may beobtained using various technologies, such as, but not limited to,retinal projection, image projection, image fusion and injection, beingdisplayed or captured with a scene camera as in presented example.Indeed, determining such characteristics could be difficult in anear-to-eye optical devices using plural optical components, such aslenses, filters, beam splitters, light guides and/or mirrors. An exampleof a near-to-eye optical device is an ocular device, such as amicroscope. A desired feature in a gaze tracking method is itsinvariance against the movement of the eye with respect to the measuringsensor.

FIG. 1A illustrates an example system in accordance with at least someembodiments of the present invention. Illustrated is a user's eye 100,which comprises the pupil 110. In the illustrated example, the user usesa microscope 120, which is an example of an ocular device or in generala near-to-eye optical device. Microscope 120 comprises a plurality ofoptical components such as lenses, filters, beam splitters, light guidesand/or mirrors, which are schematically illustrated using referencenumber 130. A sample is visible through the microscope on plate 140.Plate 140 may be of glass, for example, enabling illumination of thesample from below. Two possible gaze targets 150 are schematicallyillustrated on plate 140.

Tracking the gaze of a user of a microscope gives valuable informationconcerning how the user observes the sample under study, for examplewhat parts of the sample the user focuses on, for how long and what kindof gaze strategy the user uses. In addition, the pupil size may beestimated. This information may be used to estimate the user'sawareness, workload and level of expertise, in educational purposes, andin assisting annotation of image areas. Gaze information may be used incontrolling the functioning of an optical device, such as a microscope,for example. Thus manual controlling may be reduced.

A pathologist or laboratory worker may also use his gaze point on asample plate to guide the microscope to move the sample. As a furtherexample, a digital viewfinder may provide a visual indication when theuser is looking at a target which may be a human. Further, a sample maybe automatically associated based on the gaze point identification withmetadata indicating an extent to which it has been analysed, forexample, whether a threshold has been reached. An example is a samplewhich must be analysed at least as to 70% of its contents.

In the system of FIG. 1A, an eye-tracking camera, which is not shown forthe sake of clarity, is configured to image eye 100. The eye-trackingcamera may image eye 100 from below, wherein the eye-tracking camera maybe comprised in the ocular device being used, for example such that itmay image the eye via a semipermeable mirror in the ocular device whichis transparent to visible light but not the infra-red. The eye may beilluminated for this purpose by guide lights or light shaping or lightgenerating devices, for example visible-light or infra-red, IR, lights,which may be generated using light-emitting diodes, LEDs, for example.An advantage of IR lights is that the human eye does not detect it,making the light unobtrusive to the user. IR light also allows forfiltering out lighting and reflections of visible light sources the useris looking at or are visible in the environment, helping to control thelighting scheme. The corneal shape, illustrated in FIG. 1B, enablesextracting information on the direction the eye is turned based onglints of the lights on the moist surface of the eye. The eye-trackingsensor, such as, for example, a camera sensor, which is enabled tocapture information of the user's eye may be based at least partly oncharge-coupled device, CCD, technology, complementary metal-oxidesemiconductor, CMOS, technology, and/or photosensitive photodiodes, forexample, to produce a digital video or still data of the eye. Suchdigital video or still data of the eye may include includingreflections, glints, of the guide lights.

Likewise in the system of FIG. 1A, a scene camera—not seen in the figurefor the sake of clarity—is arranged to image plate 140. The scene cameramay likewise be based on CCD technology, for example, to produce digitalvideo or still data of the sample. The scene camera and the eye-trackingcamera may be synchronized with each other to produce time-alignedimages of eye 100 and plate 140.

In gaze tracking in general, a transformation may be sought from a 3Dcoordinate system of an eye-tracking camera to a 3D coordinate system ofa scene camera. The eye-tracking camera may be assisted by light-guidesto generate glints, as described above, to enable determining adirection where the eye is turned toward. In an ocular device, however,the presence of optical equipment 130 makes it more difficult to projecta 3D gaze point into a scene camera's 2D coordinates.

Some gaze tracking solutions are sensitive to movement of the user'shead relative to the eye-tracking camera. In these cases, if the usermoves after her calibration the gaze-tracking results will beinaccurate. Similarly, if the optical device in question moves duringoperation, it will result in a similar misalignment error. A typicalusage session of an ocular device contains constant small head movementsfor adjusting the optimal viewing angle, or head movement due to theuser leaving the operational position to view objects outside the ocularview and returning to it. The inaccuracy resulting from such headmovement hinders gaze tracking. Methods disclosed herein enable robustgaze tracking despite the afore-mentioned unwanted head movements andentering and leaving an operational position.

Much prior work in gaze tracking has used a two-dimensional, 2D, mappingfrom eye-tracking coordinates to scene camera coordinates which is lessrobust than a three-dimensional, 3D, algorithm. A 3D approach, however,is complex to implement as such because the projection between theeye-tracking camera coordinate system, where a 3D gaze point may becomputed in, and the 2D scene camera coordinate system is non-trivial todetermine due to effects of the optical equipment 130.

To overcome the difficulty in defining the transformation from thecoordinate system of the eye-tracking camera to the coordinate system ofthe scene camera, a two-phase gaze tracking method may be employed. Inthe first phase, data from the eye-tracking camera may be processed todetermine a centre of the user's pupil and the centre of the user'scornea. These will be referred to herein as the pupil centre, Pc, andcorneal centre, Cc. These may be determined as points in athree-dimensional space, enabling an optical vector L to be determinedas a normalized vector traversing these points, establishing a directionof the gaze. An optical point, Op, also referred to herein as areference point, may be determined by moving from the corneal centretowards the optical vector by the gaze distance d: Op=Cc+d*L. In someembodiments, different 3D features of the eye than the cornea and thepupil are used to compute a 3D optical point relating to the eye. Inmore general terms a first eye feature and a second eye feature may beused to obtain the optical point. An example of an eye feature otherthan the cornea and the pupil is the iris, wherefore the first andsecond eye feature may comprise the iris and the pupil, for example.

The gaze distance may be determined as follows: firstly, a virtual planemay be fitted to light-guide, such as IR LED locations. Secondly, theplane may then be shifted away from the user by a constant value, in thevirtual plane's normal direction. The constant value is guessed or it isestimated during the user calibration. Thirdly, an intersection betweenthe shifted plane and the (parameterized) vector Cc+d*L is computed,that is, d is found. Thus the gaze distance may be estimated permeasurement and may be at least slightly different each time. Other waysof determining the gaze distance are also possible. The gaze distance isthe distance from the eye to where the user's gaze is focused.

FIG. 4 illustrates an example of gaze distance estimation. A LED planeis fitted into LEDs by minimizing distances between the LEDs and theplane. To estimate the gaze distance, the location of this plane ismathematically varied such that the plane is moved away from the eye bya distance D fixed in a direction which is orthogonal to the LED planewhen it is fitted to the LED elements. This results in the plane at theviewing plane location illustrated in FIG. 4 . The distance D fixed maybe guessed, or it may be estimated during user calibration, or it may beknown based on prior knowledge. The optical point, Op, is defined as theintersection between the vector (Cc+dL) and the viewing plane.Illustrated are three possible optical vectors and the correspondingoptical points Op1, Op2 and Op3.

Once the three coordinates of the reference point are determined, the 3Dreference point may be mapped in a second phase of the overall two-phaseprocess into the two-dimensional scene camera coordinate system toobtain the gaze point, Gp, using the user's calibration matrix K:

Gp=K*Op

Calibration matrix K may be determined by conducting a calibrationprocedure with the user. The user fixates several target points with hergaze. The number of target points may be at least three or more thanthree. The target points should for ideal operation not be in a lineararrangement, for example they may form a grid of target points whichavoids the linear arrangement of the target points being disposed on astraight line. For each target point, a calibration sample is collectedfrom the eye-tracking camera which constitutes an annotated targetpoint, Gp truth, and the optical point. These calibration samples takentogether form a matrix equation from which the calibration matrix K canbe solved. A calibration reference point in the three-dimensional spacewhere the optical point will be determined corresponds to each of thetarget points. The larger the number of target points and samples, thebetter will be the expected accuracy of gaze tracking based on thecalibration matrix. While a calibration matrix is discussed herein, moregenerally calibration information may be used.

Overall, the procedure may be characterized as obtaining first andsecond eye features from a two-dimensional pixel scene of theeye-tracking camera. Examples of the first and second eye featuresinclude the corneal centre and pupil centre. From locations of the firstand second eye features in 3D space the reference point in athree-dimensional coordinate system is determined, and from thereference point the gaze point in a two-dimensional scene camera view isobtained. The coordinates of the gaze point may be determined, in thecoordinate system of the scene camera, even in the event the gaze pointis outside the view of the scene camera.

The obtaining of the corneal centre and pupil centre from the output ofthe eye-tracking camera may be performed, for example, using thephysical eye model described in Hennessey, Craig, Borna Noureddin, andPeter Lawrence. 2006. “A Single Camera Eye-Gaze Tracking System withFree Head Motion.” In Proceedings of the 2006 Symposium on Eye TrackingResearch & Applications, 87-94. ACM. Alternatively, a neuralnetwork-based method may be used where the neural network is trained tooutput the optical vector when given an eye-tracking camera image asinput.

At least some embodiments of the herein described gaze tracking processare beneficial and advantageous in that the reference point isdetermined based on the 3D characteristics of the eye, wherefore itslocation is to a degree invariant to small movements of the head.Further, alternatively or in addition, the gaze point may be determinedalso where it is disposed outside the view of the scene camera.

FIG. 1B illustrates coordinate mapping in accordance with at least someembodiments of the present invention. Eye 100 is disposed on the left,with pupil 110 and the corneal centre 160 marked thereon. The opticalvector multiplied by the gaze distance, d*L, is denoted as element 161,pointing to the reference point 170.

Reference point 170 is located in three-dimensional coordinate system,165. A mapping 180 from coordinate system 165 to two-dimensionalcoordinate system 185 of the scene camera is denoted as mapping 180 inFIG. 1B. The mapping associates reference point 170 to gaze point 190 inthe coordinate system 185 of the scene camera.

For example, in terms of a practical implementation, the ocular part(s)of a microscope may be supplemented with a module comprising theeye-tracking camera, suitable circuitry, and at least one light sourcefor structured light, such as infrared light emitting elements. If themicroscope contains an ocular part for both eyes, both ocular parts maybe provided with a similar module. In that case, the estimated gazepoint may be determined as a weighted combination of separatelydetermined left and right eye gaze points, weights being assigned scorevalues of the estimation. The microscope may also contain a scene camerathat views the sample and sees at least part of the view the user sees.A light path to the scene camera can be directed through a beam splitterwhich directs the same view to the user and to the scene camera, withdifferent optical elements.

The cameras may be connected into a computer using suitable interfaces,such as universal serial bus, USB, connectors, or integrated electricalleads. The computer may be furnished with a computer program which readsthe camera streams and estimates the gaze point in the scene cameracoordinates, as is depicted in FIG. 1B. The computer program may beequipped with a graphical user interface which, for example, may beconfigured to show the scene camera view with a superimposed estimatedgaze point. The computer program may also be configured to run theconfiguration process with the user to determine the calibration matrixK.

While described herein in terms of utilizing pupil and glint locationsand a physical eye model, the optical point may alternatively bedetermined by other means than with a physical eye model. For example,machine learning approaches based on deep convolutional networks can betaught to automatically translate pupil and glint locations into gazepoints and/or gaze directions. Where the ocular device used is a gunsight, for example, the two-dimensional output of the scene cameracorresponds to plural gaze directions where the user is gazing.

FIG. 2 illustrates an example apparatus capable of supporting at leastsome embodiments of the present invention. Illustrated is device 200,which may comprise, for example, a gaze tracking module for an oculardevice. Comprised in device 200 is processor 210, which may comprise,for example, a single- or multi-core processor wherein a single-coreprocessor comprises one processing core and a multi-core processorcomprises more than one processing core. Processor 210 may comprise, ingeneral, a control device. Processor 210 may comprise more than oneprocessor. Processor 210 may be a control device. A processing core maycomprise, for example, a Cortex-A8 processing core manufactured by ARMHoldings or a Steamroller processing core designed by Advanced MicroDevices Corporation. Processor 210 may comprise at least one QualcommSnapdragon and/or Intel Atom processor. Processor 210 may comprise atleast one application-specific integrated circuit, ASIC. Processor 210may comprise at least one field-programmable gate array, FPGA. Processor210 may be means for performing method steps in device 200, such asdetermining a reference point and performing a mapping. Processor 210may be configured, at least in part by computer instructions, to performactions.

Device 200 may comprise memory 220. Memory 220 may compriserandom-access memory and/or permanent memory. Memory 220 may comprise atleast one RAM chip. Memory 220 may comprise solid-state, magnetic,optical and/or holographic memory, for example. Memory 220 may be atleast in part accessible to processor 210. Memory 220 may be at least inpart comprised in processor 210. Memory 220 may be means for storinginformation. Memory 220 may comprise computer instructions thatprocessor 210 is configured to execute. When computer instructionsconfigured to cause processor 210 to perform certain actions are storedin memory 220, and device 200 overall is configured to run under thedirection of processor 210 using computer instructions from memory 220,processor 210 and/or its at least one processing core may be consideredto be configured to perform said certain actions. Memory 220 may be atleast in part comprised in processor 210. Memory 220 may be at least inpart external to device 200 but accessible to device 200.

Device 200 may comprise a transmitter 230. Device 200 may comprise areceiver 240. Transmitter 230 and receiver 240 may be configured totransmit and receive, respectively, information in accordance with atleast one communication standard. Transmitter 230 may comprise more thanone transmitter. Receiver 240 may comprise more than one receiver.Transmitter 230 and/or receiver 240 may be configured to operate inaccordance with global system for mobile communication, GSM, widebandcode division multiple access, WCDMA, 5G, long term evolution, LTE,IS-95, wireless local area network, WLAN, USB, Ethernet and/or worldwideinteroperability for microwave access, WiMAX, standards, for example.

Device 200 may comprise a near-field communication, NFC, transceiver250. NFC transceiver 250 may support at least one NFC technology, suchas NFC, Bluetooth, Wibree or similar technologies.

Device 200 may comprise user interface, UI, 260. UI 260 may comprise atleast one of a display, a keyboard, a touchscreen, a vibrator arrangedto signal to a user by causing device 200 to vibrate, a speaker and amicrophone. A user may be able to operate device 200 via UI 260, forexample to perform a calibration process and/or gaze trackingoperations.

Processor 210 may be furnished with a transmitter arranged to outputinformation from processor 210, via electrical leads internal to device200, to other devices comprised in device 200. Such a transmitter maycomprise a serial bus transmitter arranged to, for example, outputinformation via at least one electrical lead to memory 220 for storagetherein. Alternatively to a serial bus, the transmitter may comprise aparallel bus transmitter. Likewise processor 210 may comprise a receiverarranged to receive information in processor 210, via electrical leadsinternal to device 200, from other devices comprised in device 200. Sucha receiver may comprise a serial bus receiver arranged to, for example,receive information via at least one electrical lead from receiver 240for processing in processor 210. Alternatively to a serial bus, thereceiver may comprise a parallel bus receiver.

Device 200 may comprise further devices not illustrated in FIG. 3 .Device 200 may comprise a fingerprint sensor arranged to authenticate,at least in part, a user of device 200. In some embodiments, device 200lacks at least one device described above. For example, some devices 200may lack the NFC transceiver 250 and/or at least one other unitdescribed herein.

Processor 210, memory 220, transmitter 230, receiver 240, NFCtransceiver 250, and/or UI 260 may be interconnected by electrical leadsinternal to device 200 in a multitude of different ways. For example,each of the aforementioned devices may be separately connected to amaster bus internal to device 200, to allow for the devices to exchangeinformation. However, as the skilled person will appreciate, this isonly one example and depending on the embodiment various ways ofinterconnecting at least two of the aforementioned devices may beselected without departing from the scope of the present invention.

FIG. 3 is a flow graph of a method in accordance with at least someembodiments of the present invention. The phases of the illustratedmethod may be performed in device 200, an auxiliary device or a personalcomputer, for example, or in a control device configured to control thefunctioning thereof, when installed therein.

In phase 310, at least one eye image is obtained from the eye-trackingcamera. In phase 320, the pupil and glints of guide lights are locatedin the at least one eye image. In phase 330, a 3D pupil centre, Pc, and3D corneal centre, Cc, of the user are determined based on the pupil andglints, for example with the assistance of a model of the physical shapeof the eye. In phase 340, the optical vector L is determined, asdescribed herein above. In phase 350, the gaze distance is determined.

In phase 360, the three-dimensional reference point, also known as theoptical point, Op, is determined as Op=Cc+d*L. Finally, in phase 370,the gaze point, Gp, is obtained by mapping the optical point into thetwo-dimensional scene camera view using the calibration matrix K:Gp=K*Op. The gaze point is the point on the plate 140 the user islooking at.

It is to be understood that the embodiments of the invention disclosedare not limited to the particular structures, process steps, ormaterials disclosed herein, but

are extended to equivalents thereof as would be recognized by thoseordinarily skilled in the relevant arts. It should also be understoodthat terminology employed herein is used for the purpose of describingparticular embodiments only and is not intended to be limiting.

Reference throughout this specification to one embodiment or anembodiment means that a particular feature, structure, or characteristicdescribed in connection with the embodiment is included in at least oneembodiment of the present invention. Thus, appearances of the phrases“in one embodiment” or “in an embodiment” in various places throughoutthis specification are not necessarily all referring to the sameembodiment. Where reference is made to a numerical value using a termsuch as, for example, about or substantially, the exact numerical valueis also disclosed.

As used herein, a plurality of items, structural elements, compositionalelements, and/or materials may be presented in a common list forconvenience. However, these lists should be construed as though eachmember of the list is individually identified as a separate and uniquemember. Thus, no individual member of such list should be construed as ade facto equivalent of any other member of the same list solely based ontheir presentation in a common group without indications to thecontrary. In addition, various embodiments and example of the presentinvention may be referred to herein along with alternatives for thevarious components thereof. It is understood that such embodiments,examples, and alternatives are not to be construed as de factoequivalents of one another, but are to be considered as separate andautonomous representations of the present invention.

Furthermore, the described features, structures, or characteristics maybe combined in any suitable manner in one or more embodiments. In thepreceding description, numerous specific details are provided, such asexamples of lengths, widths, shapes, etc., to provide a thoroughunderstanding of embodiments of the invention. One skilled in therelevant art will recognize, however, that the invention can bepracticed without one or more of the specific details, or with othermethods, components, materials, etc. In other instances, well-knownstructures, materials, or operations are not shown or described indetail to avoid obscuring aspects of the invention.

While the forgoing examples are illustrative of the principles of thepresent invention in one or more particular applications, it will beapparent to those of ordinary skill in the art that numerousmodifications in form, usage and details of implementation can be madewithout the exercise of inventive faculty, and without departing fromthe principles and concepts of the invention. Accordingly, it is notintended that the invention be limited, except as by the claims setforth below.

The verbs “to comprise” and “to include” are used in this document asopen limitations that neither exclude nor require the existence of alsoun-recited features. The features recited in depending claims aremutually freely combinable unless otherwise explicitly stated.Furthermore, it is to be understood that the use of “a” or “an”, thatis, a singular form, throughout this document does not exclude aplurality.

INDUSTRIAL APPLICABILITY

At least some embodiments of the present invention find industrialapplication in tracking user gaze in ocular devices.

ACRONYMS LIST

-   2D two-dimensional-   3D three-dimensional-   CCD charge-coupled device-   CMOS complementary metal-oxide-semiconductor-   USB universal serial bus

REFERENCE SIGNS LIST

100 Eye 110 Pupil 120 Microscope 130 Optical equipment 140 Plate 150Gaze targets 160 Corneal centre 161 Vector d * L 165 Three-dimensionalcoordinate system 170 Optical point (reference point) 180 Mapping 185Coordinate system of scene camera 190 Gaze point

TECHNICAL CLAUSES

Clause 1. An apparatus comprising at least one processing core, at leastone memory including computer program code, the at least one memory andthe computer program code being configured to, with the at least oneprocessing core, cause the apparatus at least to:

determine a reference point in a three-dimensional space based at leastin part on locations of first and second features of a user's eye and onthe user's gaze distance, and

perform a mapping of the reference point into a viewed scene of anear-to-eye optical device to obtain an estimated gaze point and/or gazedirection of the user who is using the near-to-eye optical device, themapping being based at least in part on calibration informationassociated with the user.

Clause 2. The apparatus according to Clause 1, wherein the first andsecond features of the user's eye are a corneal centre and a pupilcentre.

Clause 3. The apparatus according to Clause 1 or 2, wherein theapparatus is further configured to obtain the calibration information asa calibration matrix based on a calibration process wherein the usergazes at three or more target points in sequence and a separatecalibration reference point in the three-dimensional space is determinedfor each target point used in the calibration.

Clause 4. The apparatus according to any of Clauses 1-3, wherein theapparatus is further configured to obtain the gaze distance by computingan intersection point between a predetermined virtual plane and a vectorwhich traverses the first and second features of the user's eye.

Clause 5. The apparatus according to any of Clauses 1-4, wherein thenear-to-eye optical device is a microscope, wherein apparatus isconfigured to obtain the gaze direction separately for each of theuser's two eyes.

Clause 6. The apparatus according to any of Clauses 1-5, wherein theapparatus is comprised in the near-to-eye optical device.

1. An apparatus comprising at least one processing core, at least onememory including computer program code, the at least one memory and thecomputer program code being configured to, with the at least oneprocessing core, cause the apparatus at least to: determine a referencepoint in a three-dimensional space based at least in part on locationsof first and second features of a user's eye and on the user's gazedistance, and perform a mapping of the reference point into a viewedscene of a near-to-eye optical device to obtain an estimated gaze pointand/or gaze direction of the user who is using the near-to-eye opticaldevice, the mapping being based at least in part on calibrationinformation associated with the user, wherein the near-to-eye opticaldevice comprises an ocular device, and in that the apparatus isconfigured to obtain the gaze distance by computing an intersectionpoint between a predetermined virtual plane, the virtual plane fitted tolight guide locations, and a vector which traverses the first and secondfeatures of the user's eye.
 2. The apparatus according to claim 1,wherein the first and second features of the user's eye are a cornealcentre and a pupil centre.
 3. The apparatus according to claim 1,wherein the apparatus is further configured to obtain the calibrationinformation as a calibration matrix based on a calibration processwherein the user gazes at three or more target points in sequence and aseparate calibration reference point in the three-dimensional space isdetermined for each target point used in the calibration.
 4. Theapparatus according to claim 1, wherein the near-to-eye optical deviceis a microscope, wherein the apparatus is configured to obtain the gazedirection separately for each of the user's two eyes.
 5. The apparatusaccording to claim 1, wherein the apparatus is comprised in thenear-to-eye optical device.
 6. A method comprising: determining areference point in a three-dimensional space based at least in part onlocations of first and second features of a user's eye and on the user'sgaze distance, and performing a mapping of the reference point into aviewed scene of a near-to-eye optical device to obtain an estimated gazepoint and/or gaze direction of the user who is using the near-to-eyeoptical device, the mapping being based at least in part on calibrationinformation associated with the user, wherein the near-to-eye opticaldevice comprising an ocular device, and by obtaining the gaze distanceby computing an intersection point between a predetermined virtualplane, the virtual plane fitted to light guide locations, and a vectorwhich traverses the first and second features of the user's eye.
 7. Themethod according to claim 6, further comprising obtaining thecalibration information as a calibration matrix based on a calibrationprocess wherein the user gazes at three or more target points insequence and a separate calibration reference point in thethree-dimensional space is determined for each target point used in thecalibration.
 8. The method according to any of claims wherein thenear-to-eye optical device is a microscope, wherein the method furthercomprises obtaining the gaze direction separately for each of the user'stwo eyes.
 9. The method according to claim 6, further comprising atleast one of controlling a microscope based at least partly on theestimated gaze point and/or gaze direction of the user, and identifyingthe user based on the calibration information.
 10. A non-transitorycomputer readable medium having stored thereon a set of computerreadable instructions that, when executed by at least one processor,cause an apparatus to at least: determine a reference point in athree-dimensional space based at least in part on locations of first andsecond features of a user's eye and on the user's gaze distance, andperform a mapping of the reference point into a viewed scene of anear-to-eye optical device to obtain an estimated gaze point and/or gazedirection of the user using the near-to-eye optical device, the mappingbeing based at least in part on calibration information associated withthe user, wherein the near-to-eye optical device comprising an oculardevice, and by the set of computer readable instructions beingconfigured to cause the apparatus to obtain the gaze distance bycomputing an intersection point between a predetermined virtual plane,the virtual plane fitted to light guide locations, and a vector whichtraverses the first and second features of the user's eye.